Solver Schemes for Linear Systems Oral Comprehensive Exam Position Paper

This report presents different approaches to solving sparse linear systems—direct and iterative solvers—and compares a number of methods in each category. I also consider the two techniques of using solvers for linear systems: single method solver schemes and multi-method solver schemes. I also present some of the popular systems that use these techniques. This involves surveying the relevant literature that contains in-depth information about techniques in the field. Linear systems are a form of representation for problems in a variety of domains, including but not limited to statistics, thermodynamics, electric circuits, quantum mechanics, nuclear engineering, fossil fuels, robotics and computational fluid dynamics. Solution methods in numerical optimization and PDE-based simulations frequently rely on the efficient solution of large, typically sparse linear systems. Given that linear systems are widespread [75, 74] across different areas of research, providing accurate and efficient solution methods plays a critical role for scientists in these fields. There are two classes of linear systems: sparse and dense. The systems in which most of the elements are zero are known as sparse systems. Those systems for which most of the elements are non-zero are referred to as dense linear systems. A common approach of storing sparse matrices requires only the non-zero elements to be stored, along with their location, instead of storing all the elements including the zero elements. Large sparse linear systems tend to represent real world problems better than dense systems and large sparse systems occur more frequently than small or dense systems. In this report, I focus on sparse linear systems. Large sparse linear systems arise in many computational problems in science and engineering. Over the last several decades, applied mathematicians and computer scientists have developed multiple approaches to solving such linear systems. The traditional approach involves using a single solver, possibly combined with a preconditioner to get the solution. Preconditioning is discussed in more detail in Section 4. This solver can be chosen among a number of available options. For sparse linear systems, iterative solvers would be a more reasonable choice than direct solvers [30] because direct solvers can be more computationally expensive and might not even produce a solution. The challenge then is to identify which solver to use among the numerous options available because it is nontrivial even for experts to determine which is the best solver. The main reason is that the best solution is not consistent for a variety of problems occurring in different domains, or even different problems from the same domain. In addition, given that these are iterative solutions, which are